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DETAILED ACTION 

1 . This action is responsive to communications: amendment filed 10/12/05 to the 
application filed. on 4/26/01. 

2. Claims 1, 4, 8, 10-20 are pending in the case. Claims 1 and 15 are independent 
claims. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

5. Claims 1 , 4, 1 1-12, 14-20 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Stifelman et al., The Audio Notebook, Paper and Pen Interaction with 
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Structured Speech, SIGCHI'S 01, March 31 -April 4, 2001, vol. 3, Iss. 1, ACM 2001, 
pages 182-189, in view of Arons et al. (US Pat No. 6,529,920 B1, 3/4/03, filed 3/5/99). 

Regarding independent claim 1 , Stifelman discloses: 

- receiving a notation from a notetaking user during a meeting (page 182, 
Abstract, Introduction, page 183, Audio Notebook Version 1: synchronizing 
user's handwritten notes during a meeting with a digital audio recording and 
indexing the written notes and audio indicate receiving notes from a user during a 
meeting) 

- automatically recording an index value for the notation, the index value based on 
the context of the notation (page 182, Introduction: "The Audio Notebook 
synchronizes the user's handwritten notes with a digital audio recording. The 
user's natural activity - writing and page turns - implicitly indexes the audio for 
later retrieval page 183, Prior Work in Indexing Audio : "The AIR project 
(Activity-based Information Retrieval) proposed employing user activity (e.g. 
notetaking, writing on whiteboards, user location) to index multimedia data ... 
Audio Notebooks links audio recording to notes taken on paper and provides 
several techniques to access the audio ... Dynomite indexes audio with 
notetaking activity on a pen-based computer. Users can manually assign 
keywords to pages of notes ...the audio and video are indexed by all notes 
written on, or beamed to, the LiveBoard, and by pages changes on the 
LiveBoard") 
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- receiving a quantity of multimedia information from at least one multimedia 
source (page 183, Prior Work in Indexing Audio: the fact that the multimedia 
data as audio and video are indexed by all notes written, implies that a quantity 
of multimedia information from at least one multimedia source, audio source and 
video source, is received) 

- automatically selecting at least one portion of the quantity of the multimedia 
information based on the index value of the notation (page 183, Prior Work in 
Indexing Audio: "The audio and video are indexed by all notes written on, or 
beamed to, the LiveBoard, and by page changes on the LiveBoard page 
183, Audio Notebook Version 1: "An early version of the Audio Notebook [7] 
demonstrated the basic concept of linking notes on paper with an audio 
recording. This early prototype showed the concept of automatic page number 
detection, and selecting on the page to begin audio playback (Fig. 1) ... Playback 
begins at the point in the audio recording that corresponds to when the note was 
originally written ... Dragging the pen along an audio scrollbar navigates a 
timeline of the audio associated with each page pages 183-184, Audio 
Scrollbar with Audio Cursor: "when a user selects somewhere on a page to 
begin playback, the audio cursor lights up showing the corresponding location in 
the timeline .. ") 

- automatically creating an association between the notation and the selected 
portion of the quantity of multimedia information, where the association enables 
access to the selected portion of the quantity of multimedia information (page 
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182, Introduction : synchronizing between the user's written notes and the audio 
recording and indexing the recorded audio based on the user's notetaking for 
later retrieval of audio show an association between the notation and the 
correspondent audio where the association enables access to the selected 
multimedia for retrieval; page 183, Prior Work in Indexing Audio: "the Audio 
Notebook links audio recording to notes taken on paper and provides several 
techniques to access the audio \ "Classroom 2000 [1] captures audio, video, and 
slides, and links it to notes taken on tablet computers and electronic whiteboards. 
Access to the captured material is through an HTML-based web browser"] page 

1 83, Audio Notebook Version 1 : "An early version of the Audio Notebook [7] 
demonstrated the basic concept of linking notes on paper with an audio 
recording. This early prototype showed the concept of automatic page number 
detection, and selecting on the page to begin audio playback" ) 

- storing the notation and the association for retrieval at a future time, where the 
future time is one of a time during the meeting and a time after the meeting 
(page 182, Introduction : Synchronizing between the user's written notes and 
the audio recording and indexing the recorded audio based on the user's 
notetaking for later retrieval of audio show that the notation and the 
corresponding audio recorded are stored for later retrieval; page 183, Audio 
Notebook Version 1: playback the recorded audio based on the selected page 
of the taken notes indicates that the notation taken by a user and the association 
of the written notes and the recorded audio are stored for retrieval later at a 



Application/Control Number: 09/843, 1 97 Page 6 

Art Unit: 2178 

future time; the later retrieval implies that retrieval can occur any time after 
recording data, and thus further implies that "later" or the future time is one of a 
time during the meeting and a time after the meeting; page 183, Audio 
Notebook Version 1 : the fact that after recording the audio of a lecture or 
meeting based upon the notetaking by a user, the audio can be accessed by 
space or by time indicates that retrieving the recorded audio can be made any 
time, i.e. during the meeting, as long as it occurs after the audio recording; in 
other words, the future time can be during the meeting) 
- wherein a single action by the notetaking user initiates the steps of receiving the 
notation, recording, selecting, creating and storing (page 182, Introduction, 
page 183, Audio Notebook Version 1: as mentioned above, Stifelman 
discloses that the user's natural activity, such as writing and page turns, implicitly 
indexes the audio for later retrieval, the audio and video are indexed by all notes 
written on, or beamed to, the LiveBoard, and Playback begins at the point in the 
audio recording that corresponds to when the note was originally written; the fact 
that the audio and video are indexed when the note was originally written, linking 
audio recording to notes taken on paper, and playback the recorded audio based 
on the selected page of the taken notes inherently indicates that based on 
receiving a note written by a user, the index is recorded corresponding to the 
selected audio and video where said index recording shows the creation of the 
association between the taken notes and related multimedia which is stored for 
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playback and retrieval later on; in other words, the notetaking of a user initiates 

the steps of recording, selecting, creating, and storing) 
Stifelman does not disclose that the future time for retrieving the stored notation and the 
association is a time during the meeting. 

Arons discloses that the future time for retrieving the stored notation and the association 
is a time during the meeting (col 3, lines 30-67: the fact that users can select any word, 
drawing, or mark on a page to instantly cue playback to the time around when the mark 
was made where marking was made on the page of the notation during the meeting, 
interview, lectures, in-person conversation, etc and the playback implies that the stored 
notation and multimedia is retrieved to be played). 

It would have been obvious to an ordinary skill in the art at the time of the invention was 
made to have combined Arons into Stifelman since Arons teaches that the future time 
for retrieving the stored notation and the association is during the meeting providing the 
advantage to incorporate into Stifelman for offering to users more capabilities to retrieve 
the stored notation and the associated multimedia during the meeting instead of after 
the meeting only. 

Regarding claim 4, which is dependent on claim 1 , Stifelman discloses that the quantity 
of multimedia information includes information for accessing a second quantity of 
multimedia information (page 183, Prior Work in Indexing Audio: the fact that the audio 
and video are indexed by all notes written on, or beamed to, the LiveBoard indicates 
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that beside the audio, video is the second multimedia information to access while taking 
notes). 

Regarding claim 11, which is dependent on claim 8, Stifelman discloses: 

- receiving a quantity of information from a user (page 1 86, Use of the Audio 
Recordings: the fact that a student adds information to her notes shows that the 
information is received from a user) 

- revising at least one of the stored notations and its respective association in 
response to the quantity of information received from the user (page 186, Use of 
the Audio Recordings: marking in the notes, writing something in the notes, and 
adding more details into the notes by a user are actions to rewse the stored 
notations and its respective association in response to the information such as 
text written to notes or details added to the notes received from a user) 

Regarding claim 1 2, which is dependent on claim 1 1 , Stifelman discloses that the 
quantity of information received from a user includes a copy of at least a portion of the 
plurality of notations, where the user has altered at least one of the plurality of notations 
to indicate the desired revision (page 186, Review Session: a user can add a few 
annotations to a note such as putting star symbols next to important areas or marking 
things in a note to review where the star symbols or the mark made by a user have 
altered at least one of the plurality of notes to indicate the desired revision). 
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Regarding claim 14, which is dependent on claim 13, Stifelman discloses that the 
quantity of information received from a user includes a copy of at least a portion of the 
plurality of notations, where the user has altered at least one of the plurality of notations 
to indicate the desired revision (page 186, Review Session: a user can add a few 
annotations to a note such as putting star symbols next to important areas or marking 
things in a note to review where the star symbols or the mark have altered at least one 
of the plurality of notes to indicate the desired revision). 

Claims 15-16 are for an apparatus of method claim 1 , and are rejected under the same 
rationale. 

Claims 17-18 are for an apparatus of method claims 11-12, and are rejected under the 
same rationale. 

Claims 19-20 are for an apparatus of method claims 6-7, and are rejected under the 
same rationale. 

6. Claim 8 remains rejected under 35 U.S.C. 103(a) as being unpatentable over 
Stifelman in view of Arons as applied to claim 1 above, and further in view of Davis et 
al., NotePals: Lightweight Note Sharing by the Group, for the Group, ACM 1999, pages 
338-345 (IDS submitted by Applicants). 
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Regarding claim 8, which is dependent on claim 1, Stifelman discloses that the steps of 
receiving a notation, recording, receiving a quantity of information, selecting, creating, 
and storing are repeated for a plurality of notations (page 183, Prior Work in Indexing, 
Audio Notebook Version 1 : fhe fact that the audio and video are indexed when the 
note was originally written, linking audio recording to notes taken on paper, where the 
note, audio and video data are a quantity of information are received, and playback the 
recorded audio based on the selected page of the taken notes inherently indicates that 
based on receiving a note written by a user, the index is recorded corresponding to 
the selected audio and video where said index recording shows the creation of the 
association between the taken notes and related multimedia which is stored for 
playback and retrieval later on; page 183, Prior Work in Indexing Audio: the fact that 
the audio and video are indexed by all notes written on, or beamed to, the LiveBoard, or 
capturing audio, video, and slides and linking it to notes t aken on tablet computers and 
electronic whiteboards indicates that the claimed steps are repeated for a plurality of 
notations). 

Stifelman does not disclose transmitting the plurality of notations and their respective 
association via an electronic network to at least one user for future retrieval by said 
user. 

However, Stifelman does teach the linking of captured multimedia and the notes taken 
on tablet computers and electronic whiteboards and access to the captured multimedia 
is through an HTML-based web browser (page 183, Prior Work in Indexing Audio). 
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Davis discloses the group members in a meeting can retrieve and view the notes taken 
with browsers (page 338, abstract) and automatically capturing notes taken in any 
context and making those notes and the related documents accessible to an entire 
workgroup via the web (page 338, Introduction). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Davis into Stifelman for the following reason. Davis 
discloses the ability to access notes to an entire workgroup via the web and retrieving 
notes during a meeting using the web browser providing the advantage to incorporate 
into accessing the captured multimedia related to the notes taken in Stifelman via the 
web browser for transmitting the notes and their respective association, which are the 
related multimedia of the notes, via an electronic network, which is the web, to groups of 
users instead of limiting the use of taken notes and related audio and video in a meeting 
by a user. 

7. Claims 10 and 13 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over Stifelman in view of Arons and Davis as applied to claims 1 and 9 above, and 
further in view of Mora et al. (US Pat No. 6,161,1 13, 12/12/00, 1/20/98, priority 1/21/97). 

Regarding claim 10, which is dependent on claim 8, Stifelman, Arons and Davis do not 
disclose that a plurality of the notations and their respective associations are transmitted 
via an electronic mail message. 
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Mora discloses sending the meeting minutes to the attendees of the meeting via email 
(col 14, lines 34-52). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Mora into Stifelman, Arons and Davis since Mora can 
send the meeting minutes to the attendees of the meeting via email providing the 
advantage to incorporate into Stifelman, Arons, and Davis for rapidly distributing notes 
taken and associated multimedia in the meeting to the meeting attendees. 

Regarding claim 13, which is dependent on claim 8, Stifelman discloses that the 
quantity of information received from a user includes a copy of at least a portion of the 
plurality of notations, where the user has altered at least one of the plurality of notations 
to indicate the desired revision (page 186, Review Session: a user can add a few 
annotations to a note such as putting star symbols next to important areas or marking 
things in a note to review where the star symbols or the mark have altered the at least a 
portion one of the notes). 

Stifelman does not disclose that in step transmitting, the plurality of notations and the 
respective notations are transmitted as an electronic mail message via an electronic 
mail network, the electronic mail message containing a predetermined electronic mail 
address, and in the step of receiving a quantity of information from a user, the quantity 
of information is received via the predetermined electronic mail address. 
Mora discloses sending the meeting minutes to the attendees of the meeting via email 
(col 14, lines 34-52). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Mora into Stifelman and Davis since Mora can send the 
meeting minutes to the attendees of the meeting via email providing the advantage to 
incorporate electronic mail network for sending the notes taken and the associated 
multimedia since the meeting minutes, which are records of a meeting, thus include also 
the notes taken during the meeting as in Stifelman and Davis. In addition, the fact that 
Mora uses the email system for sending meeting minutes to the attendees of the 
meeting suggests that the email receiver have a predetermined electronic mail address. 



Response to Arguments 

8. Applicant's arguments filed 10/12/05 have been fully considered but they are not 
persuasive. 

Applicants address that Stifelman is not available to be combined with Arons in the 
claim 1, 4, 11-12, 14-20 rejections in view of the submission of the Declaration 
(Remarks, page 7). 

However, the claim rejection remains since the submitted declaration is ineffective to 
overcome the Stifelman reference as mentioned in the Response to the Amendment 
below. 

Response to Amendments 

9. The declaration filed on 10/12/05 under 37 CFR 1.131 has been considered but 
is ineffective to overcome the Stifelman reference. 
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The declarations were not signed by all the co-inventors and a petition under 37 CFR 
1 .47 in this application was not granted. MPEP 715.04 (A-D) states: 

"The following parties may make an affidavit or declaration under 37 CFR 1. 131: 

(A) All the inventors of the subject matter claimed, 

(B) An affidavit or declaration by less than all named inventors of an application is 
accepted where it is shown that less than all named inventors of an application 
invented the subject matter of the claim or claims under rejection. For example, 
one of two joint inventors is accepted where it is shown that one of the joint 
inventors is the sole inventor of the claim or claims under rejection. 

(C) **> If a petition under 37 CFR 1.47 was granted or the application was 
accepted under 37 CFR 1.42 or 1.43, the affidavit or declaration may be signed 
by the 37 CFR 1.47 applicant or the legal representative, where appropriates . 

(D) The assignee or other party in interest when it is not possible to produce the 
affidavit or declaration of the inventor. Ex parte Foster, 1903 CD. 213, 

105 O.G. 261 (Comm'rPat 1903). 

Affidavits or declarations to overcome a rejection of a claim or claims must be made 
by the inventor or inventors of the subject matter of the rejected claim(s), a party 
qualified under 37 CFR 1.42, 1.43, or 1.47, or the assignee or other party in interest 
when it is not possible to produce the affidavit or declaration of the inventor(s). Thus, 
where all of the named inventors of a pending application are not inventors of every 
claim of the application, any affidavit under 37 CFR 1. 131 could be signed by only the 
inventor(s) of the subject matter of the rejected claims. Further, where it is shown that a 
joint inventor is deceased, refuses to sign, oris othenvise unavailable, the signatures of 
the remaining joint inventors are sufficient However, the affidavit or declaration, even 
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though signed by fewer than all the joint inventors, must show completion of the 
invention by all of the joint inventors of the subject matter of the claim(s) under rejection. 
In re Carlson, 79 F.2d 900, 27 USPQ 400 (CCPA 1935)" 
As stated above, a petition under 37 CFR 1.47 is required, 37 CFR 1.47 states in part: 
11 The oath or declaration in such an application must be accompanied by a petition 
including proof of the pertinent facts, the fee set forth in § 1.17(h), and the last known 
address of the nonsigning inventor" 
Applicants should consider filing the declarations and petition by the assignee of the 
invention, in the case where it is not possible to have the signatures of all the co- 
inventors. See MPEP 715.04(D). 

In the interest of compact prosecution, the Examiner has considered the content of the 
declarations below: 

"In, general, proof of actual reduction to practice requires a showing that the apparatus 
actually existed and worked for its intended purpose. However, "there are some devices 
so simple that a mere construction of them is all that is necessary to constitute reduction 
to practice." In re Asahi/America Inc., **>68 F.3d 442, 37 USPQ2d 1204, 1206< (Fed. 
Cir. 1995) (Citing Newkirk v. *>Lulejian<, 825 F.2d 1581, 3USPQ2d 1793 (Fed. Cir. 
1987) and Sachs v. Wadsworth, 48 F.2d 928, 929, 9 USPQ 252, 253 (CCPA 1931). 
The claimed restraint coupling held to be so simple a device that mere construction of it 
was sufficient to constitute reduction to practice. Photographs, coupled with articles and 
a technical report describing the coupling in detail were sufficient to show reduction to 
practice.)" (MPEP 715.07 III) 
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"For an actual reduction to practice, the invention must have been sufficiently tested to 
demonstrate that it will work for its intended purpose, but it need not be in a 
commercially satisfactory stage of development If a device is so simple, and its 
purpose and efficacy so obvious, construction alone is sufficient to demonstrate 
workability. King Instrument Corp. v. Otari Corp., 767 F.2d 853, 860, 226 USPQ 402, 
407 (Fed. Cir. 1985). For additional cases pertaining to the requirements necessary to 
establish actual reduction to practice see DSL Dynamic Sciences, Ltd. v. Union Switch 
& Signal, Inc., 928 F. 2d 1122, 1126, 18 USPQ2d 1152, 1155 (Fed. Cir. 1991) ("events 
occurring after an alleged actual reduction to practice can call into question whether 
reduction to practice has in fact occurred"); Corona v. Dovan, 273 U.S. 692, 1928 CD. 
252 (1928) ("A process is reduced to practice when it is successfully performed. A 
machine is reduced to practice when it is assembled, adjusted and used. A manufacture 
[i.e., article of manufacture] is reduced to practice when it is completely manufactured. A 
composition of matter is reduced to practice when it is completely composed. " 1928 CD. 
at 262-263 (emphasis added).); Fitzgerald v. Arbib, 268 F.2d 763, 765-66, 122 USPQ 
530, 531-32 (CCPA 1959) ("the reduction to practice of a three-dimensional design 
invention requires the production of an article embodying that design" in "other than a 
mere drawing")" (MPEP 2138.05). 
Exhibits A-C, submitted as computer files, do not constitute an actual reduction to 
practice since it is not clear how these files are related to the claimed limitations. The 
Declaration does not show the mapping of the claims and the features in the exhibits A- 
C to establish the possession of the invention. Statements "the prototype embodiment 
of the invention contained all the elements of claim 1 as described and claimed in the 
above U.S. patent application" or "the prototype demonstrated to the meeting embodied 
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meeting embodied all elements of claim 1 of the invention described and claimed in the 
above U.S. patent application" are so vague statements. Only the filling of a US patent 
application which complies with the disclosure requirement of 35 USC 112 constitutes a 
constructive reduction to practice. Accordingly, Applicants have not established prior 
invention. The rejection, thus, is maintained. 

Conclusion 

10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Schmidt et al. (US Pat No. 6,418,457, 7/9/02, filed 12/10/97). 

Mcintosh (US Pat App Pub No. 2002/0010708, 1/24/02, priority 10/21/98). 
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Nicastro et al. (US Pat App Pub No. 2002/00731 14, 6/13/02, priority 10/30/00). 
Cantwell et al. (US Pat App Pub No. 2001/0034738, 10/25/01 , priority 2/22/00). 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 571-272- 
4125. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-4125. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




Cong-Lac Huynh 
Primary Examiner 
Art Unit 2178 
12/19/05 



